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. Within the Department of Health, Education and Welfare, my 
office has the responsibility for assessing the effectiveness of 
all major programs. The evidence on compensatory education has 
been a subject of paramount interest to this office for seme 
years. We know that there are millions of children who need help 
--we have sought systematically to learn how to help them. 

The debate concerning the effectiveness of compensatory edu- 
cation has been long and intense. The President's initiative of 
March 1972 has brought about renewed public debate on the subject. 
This analysis and review of the evidence concerning the effective- 
ness of compensatory education has been done in order to provide 
a basis for a more informed and complete discussion of a complex 
issue. This work has been done under severe time pressures, and 
is subject to those limitations. 

The Project Monitor for this effort was Dr. Constantine Menges, 
Assistant Director (Planning) of the Office for Civil Rights. Col- 
leagues with him in this effort have been Dr. Joan Bissell, Office 
of the Assistant Secretary for Planning and Evaluation; Dr. John 
Evans, Assistant Commissioner for Planning and Evaluation, Office 
of Education; Ms. Linda McCorkle, Office for Civil Rights; Ms. Ruth 
McVay, Office for Civil Rights; and Mr. P. Michael Timpane, Direc- 
tor of Education Planning, Office of the Assistant Secretary for 
Planning and Evaluation. This work could not have been completed 
in the short time available without the efficient and cheerful 
secretarial support of Ms. Penny Al-Rawi and Ms. Belinda Hood. 




Laurence E. Lynn, Jr. 
Assistant Secretary for 
Planning and Evaluation 
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THE EFFECTIVENESS OF COMPENSATORY EDUCATION 
SUMMARY AND REVIEW OF THE EVIDENCE 



INTRODUCTION 



In his message to Congress concerning the Equal Educational Opportu- 
nities Act of 1972 President Nixon proposed a new Federal emphasis on 
compensatory education to help disadvantaged children. The President 
proposed that this effort be focused on basic learning skills at re- 
latively high dollar levels (about $300 per child), to result in 
improved academic achievement for students in schools with high con- 
centrations of poor children. 

In forwarding his program, the President acknowledged that the steps 
he proposed represented one judgment--carefully considered but a 
judgment nevertheless- -in an area where empirical evidence concerning 
the probabilities and determinants of success or failure was imperfect. 
As his message stated: 

"For some years now, there has been a running debate 
about the effectiveness of added spending for pro- 
grams of compensatory or remedial education* Some 
have maintained there is virtually no correlation 
between dollar input and learning output; others have 
maintained there is a direct correlation; experience 
has been mixed." 

Based on careful assessment of available evidence, the President's 
judgment was also clearly stated: 

"While there is a great deal yet to be learned about 
the design of successful compensatory programs, the 
experience so far does point in one crucial direction: 
to the importance of providing sufficiently concentrated 
funding to establish the educational equivalent of a 
'critical mass,' or threshold level. Where funds have 
been spread too thinly, they have been wasted or dissi- 
pated with little to show for their expenditure* Where 
they have been concentrated, the results have been fre- 
quently encouraging and sometimes dramatic*" 

The President's proposals involve, therefore, an affirmative answer to 
the following two questions: 

. Can compensatory education be made to work? 
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. Does the application of concentrated compensatory re** 
sources (usually at higher dollar costs) in basic 
learning programs enhance the probability of success 
in compensatory education? 

This analysis is intended to amplify in additional detail our affirma- 
tive replies to these questions and set forth some honest and prudent 
qualifications which should be attached to these "yes" answers. 

Secretary Richardson testified at length before the Congress on this 
subject on March 24 and 27. But we feel that an additional review of 
the evidence is needed at this time, because the debate on the 
President's proposals for a new compensatory education effort has been 
fragmentary and incomplete. 

With respect to both questions, wo have been speaking in terms of prob- 
abilities. We do not find in the evidence any sure-fire techniques 
whereby compensatory education will always work. Similarly, the rela- 
tionship between the intensity of resources (i.e., cost3) and achieve- 
ment results means only that a "critical mass" may increase the prob- 
ability of success. There is no guarantee of success if more funds are 
spent; and very expensive programs have, in fact, failed. More than re- 
sources are necessary for a good compensatory education program; careful 
innovative planning and effective management are also essential. At the 
same time, we know that unusual combinations of circumstance and imagi- 
nation have and no doubt will produce successful compensatory education 
with relatively small per pupil costs. 

All this we have said before. But it obviously merits repeating at each 
stage of the public discussion of these issues. 

In reviewing the evidence that follows, it is also important to maintain 
a perspective concerning the maximum contribution which we expect schools 
to make in lifting the burden of children's economic and cultural depri- 
vation. Differing perspectives on this point are among the most important 
factors contributing to the divergent conclusions which reasonable men 
have reached in reviewing the evidence. Some would hold that schools can 
do little or nothing to overcome a poor child's personal history and en- 
vironment, while others would hold that schools can overcome almost all 
obstacles to learning for all children. Our assumption is that if schools 
can produce improvements in learning for disadvantaged children of even a 
relatively modest order, this both constitutes success when weighed against 
the formidable challenge to the schools which these other conditions pre- 
sent and warrants further support and exploration. 

* 

The difficult question for national policy, then, is whether the net 
weight of the evidence allows us to conclude that compensatory education 
programs can be made to help reasonable numbers of disadvantaged children 
to learn more effectively. The levei of one's expectations for success 
are presently a part of any individual's answer to this question--as much 
as or more than the weight of the evidence summarized below. 
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PROFILE OF AVAILABLE EVIDENCE 

We have tried to gather and analyze all the evaluations of compensatory 
education efforts in reading and mathematics which included pre-treatment 
and post-treatment standardized achievement test data. Our survey of 
the sources has been complete to the limits of our knowledge and time, 
and this report includes all the evidence, positive and negative, avail- 
able to the Department. There are, no doubt, flaws in some degree or 
other in most of the pieces of evidence we have considered. 



This evidence can be divided into two categories: large-scale evalu- 

ations which include considerable numbers of children from a wide variety 
of settings— notably, the national, State and city evaluations of programs 
under Title I of the Elementary and Secondary Education Act of 1965; and 
situational evidence which is limited to specific projects, schools or 
smaller research studies. 
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No one piece of evidence is either sufficiently representative of national 
compensatory education programs or sufficiently definitive in its findings 
for policymakers to make conclusions based upon it alone. Rather, array- 
ing the evidence available is more like fitting together the pieces of a 
mosaic and considering the overall pattern that emerges. 1 / 

CAN COMPENSATORY EDUCATION BE MADE TO WORK? 

The appendices of this analysis constitute our review of the best national, 
State and local evidence that we could find. As presented, these reviews 
set forth with appropriate qualifying commentary the evidence which has 
led us to conclude that the concentrated compensatory education program 
proposed by the President is a sound investment for the Nation at this 
time. It is in the nature of the evidence we review that unequivocal 
findings, negative and positive, are equally rare. 

Evidence for the effectiveness of compensatory education 
The evidence presented in the appendices covers more than 16 large-sea 1 ^ 
evaluations performed by Federal, State and local authorities concerning 
their compensatory education programs. It also reviews available evidence 
concerning specific projects where compensatory education succeeds, pre- 
liminary results from the evaluation of the national Follow Through Program 
and less comprehensive State evaluations. 

The studies covered are not of equal importance in terms of their repre- 
sentativeness , thoroughness, comprehensiveness, or presumed validity. 
Nevertheless, the drift of the evidence seems to be unmistakable; that 
compensatory education often enhances the achievement of poor children. 



JL/ A more extended discussion of the evidence will be found in the section 
on "Introduction to The Evidence." 
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Expert experience suggests that .7 grade equivalent per year is usually 
the most which disadvantaged children gain in one year of school. But 
in many of the compensatory education programs we discuss, sizabl pro- 
portions (often a majority) of the poor children tested seem to be achiev- 
ing at a greater rate than this; while a smaller but still significant 
percentage are achieving at or above the national norm (1.0 grade equiva- 
lent gain per year). 



The most complete data are those available from the State of California.—/ 
California has collected pre-treatment and post -treatment standardized 
test achievement data on children receiving Title I services for the last 
four years. Achievement data was collected for about 807. of all partici- 
pants in compensatory reading programs and analyses were conducted using 
data covering about 50% of the participating children. Only that achieve- 
ment data which met specified quality control criteria was included. Over 
the four years covered by the data, 547. to 67% of children receiving com- 
pensatory services showed a rate of reading achievement gain larger than 
the usual maximum for disadvantaged children. Analysis and resuits for 
mathematics were similar and even slightly better. We judge this to be 
clear evidence of success. 

The evidence from Colorado, Connecticut, Rhode Island, and Wisconsin is 
also indicative of success, but less comprehensive. Moreover, summary 
evidence from many of the remaining State reports, which are summarized 
in Appendix H, and which we have not had an opportunity to review, point 
in identical directions. In only a few States — notably Texas and Louisiana 
among those reviewed --does it appear that compensatory education had neg- 
ligible or very minor effects. 



The results of Title I in Los Angeles, California, pose a more difficult 
problem of judgment. There, only 17% to 32% of the children in the ele- 
mentary grades receiving compensatory services improved their rate of 
achievement gain above that of the average disadvantaged child without 
assist ance .—'Yet, we are informed that those children in the Los Angeles schools 
represented the most severe form of multiple economic and social disadvan- 
tagement, and this leads us to judge these very modest results as' indi- 
cating relative "success" under the circumstances. 



The evidence available to us concerning specific Title I projects, taken 
together with the Los Angeles evidence, demonstrates that successful 
compensatory education in settings of urban poverty poses a more difficult 
but not an impossible challenge. For example, among the more than 20 
successful compensatory education projects identified by a research effort 
which sought to discover exemplary programs, many were inner city efforts 
enrolling large proportions of disadvantaged and minority children. £/ 



1/ See Appendix B. 
2/ See Appendix I. 
2/ See Appendix p. 
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The largest of these, the More Effective Schools program In New York, 
typifies the limited success of compensatory education 
in such environments. In the More Effective Schools program, average 
per pupil achievement gains often exceeded the upper gain rates for 
disadvantaged children, but usually fell short of the rate expected 
of the average American child.. 



Preliminary evidence available in the evaluation of Follow Through shows 
much the same pattern. * The Follow Through evaluation is the most rigor- 
ous in design of all evaluations of compensatory education; but it: has not 
been completed. The following are the preliminary conclusions which can 
be drawn from the data at this time: children receiving compensatory 

help show very small but consistent improvement in learning compared 
with matched children without conpensatory help; and, the more disadvan- 
taged the children are, the more effective this compensatory help has been 
in improving their academic performance compared with similar children not 
receiving any help. 



Evidence against the effectiveness of compensatory education 

The most pessimistic view of compensatory education is that it has not 

worked and probably cannot be successful with disadvantaged children. 

The evidence we have just discussed contradicts this totally negative 
conclusion. Yet there is considerable evidence indicating that many com- 
pensatory education programs can be assessed as unsuccessful either be- 
cause too few children improved their academic performance or the rates 
of improvement did not exceed that typical for poor children. Both large 
scale and situational evaluations contain evidence of this kind. 

There have been at least three large-scale evaluations which have con- 
cluded that Title I was not successful: 

. An evaluation undertaken by 6. E. Tempo analyzed compensa- 

| tory programs conducted under Title I in 11 large cities 

j during school years 1965-67. The study found only slight 

> evidence that the program enhanced achievement on average, 

| and some clear instances where the children receiving ser- 

j vices had actually fallen farther behind. 2 



I 
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. Two national evaluations of Title I have been conducted 
under auspices of the U. S. Office of Education. ^ Both 
of these evaluations were undertaken, not by on-site 
investigators, but through a Federal-State-local infor- 
mation reporting system and concluded that Title I par- 
ticipants had showed no improvement in achievement gains 
compared with nonparticipants. 



1 / See Appendix N 
2 / See Appendix M 
3/ See Appendix M 
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In a technical sense , we have no dispute with the findings of these 
studies. However, we would note that the first study reviewed a strik- 
ingly unrepresentative sample of projects in the initial stages of 
Title I's implementation, and that the two USOE evaluations were severely 
limited and have the same limits as the State reports since they are de- 
rived from an information system rather than an on-site review. Also, 
they are severely limited in representativeness on this issue because they 
were marked by very high (over 90%) non-response rates for usuable achieve- 
ment data. Furthermore, we know that the funds under this national Title 
I program were , on the average , spread very thinly among many students and 
that the average child received no more than one or two hours per month 
assistance in reading. 

Yet another body of evidence against the effectiveness of compensatory 
education is the national evaluation of the Head Start program which 
concluded that full year Head Start programs produced cognitive gains 
for a small proportion of participants but not for most; and that the 
gains dissipated rapidly when the children entered normal school 
programs. 1 / This study was a well designed and well implemented one, and 
its findings should be accepted. We would note, nevertheless, that Head 
Start is concerned with pre-school, rather than school programs and that 
little is known concerning the extent to which Head Start programs attempted 
to concentrate on cognitive improvements for the participating children. 

In addition, the children showing most gains were minority children in 
cities. The schools which the children entered after Head Start were not 
necessarily able to provide them with the enriched learning environment 
needed to preserve the achievement gains; we consider that effective com- 
pensatory education will require more than a single year's effort for most 
disadvantaged children. 

The situational evidence against compensatory education is mote 
impressioni8tic--there are relatively few validated failures of compensa- 
tory programs but there are many, many instances where dose investigations 
of claimed success showed that the evidence was lacking or unreliable. 
Undoubtedly, many specific efforts labled as compensatory education at all 
levels of cos t and intensity have failed. 

The evidence indicating that compensatory education has not wotked is, 
we judge, sobering but not overwhelming, a counsel of caution but not of 
despair. 

THE COLEMAN REPORT: WHAT DOES IT SAY? 

The findings of the Coleman report have been a keystone for many of the 
arguments that compensatory education cannot work. The Coleman report 
was a study of the relationship between achievement and a variety of social, 



1 / See Appendix 0 
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regional and educational resource factors. The data from this analysis 
have been reanalyzed by a number of scholars in the years since 1966. 

One clear finding of the Coleman report is that variation in school 
spending within the range of existing educational practice was not a 
significant factor in explaining differences in the achievement of 
children. 

The Coleman report also suggests that there are modest educational gains 
associated with the attendance of minority group children in classes and 
schools which are racially and socio-economica lly integrated .^1/ This 
conclusion is accepted and is not at issue here. We have not presented 
compensatory education as an alternative to desegregation, but rather as 
a complementary policy .2/ It is our view that both before, during and 
after transfer from racially and/or economically isolated environments 
to more favorable learning centers, educationally disadvantaged children 
can benefit from compensatory education services. 

In this connection, it is worth noting two additional features of the 
Coleman report: 

. As the author has recently pointed out, the Coleman report 
should not be used to claim that physical desegregation is 
the only educational treatment that can have any positive 
achievement effect 8.3 

• There is no direct evidence in the Coleman report for the 
conclusion, sometimes drawn from it, that compensatory edu- 
cation does not work. The Coleman report analyzed the 
existing range of school conditions in 1965-66 and had 
nothing to say about situations in which very substantial 
additional resources above normal school expenditures were 
provided for bas ic learning programs . The Coleman report 
did not analyze any such intensive programs. 
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V See Source 79, 

2/ See Appendix T % 

3/ New York Times. April 9, 1972. 






( 



a 



11 



EVIDENCE CONCERNING THE RELATIONSHIP BETWEEN LEVEL OF EDUCATIONAL BE 

SOURCES AND ACHIEVEMENT GAINS 
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With ro ect to the first question on whether or not compensatory educa- 
tion Ci.iL work, the evidence, as we have noted. Is definitely encourag- 
ing. The Important difference between success and non-success appears 
to depend on wlwcher compensatory education funds have been channeled 
Into traditional patterns of expenditure— salary Increases, routine tech- 
niques, ; etc. — or whether they have been used to develop supplementary, 
focused, compensatory education programs. The reason there Is so much 
evidence of failure Is that resources have more often been used In the 
former rather than the latter manner. 

On the second question of how closely effective compensatory education 
is related to increased expenditures, the evidence, and therefore our 
conclusion, is much less clear. However, on the basis of the common 
sense observation that a supplementary compensatory education program 
will require additional resources, on the evidence that the elements 
of programs found to be successful require significant additional 
resources (e.g.. Individualized instruction), and on the basis of 
some fragmentary evidence from several studies which have attempted 
to relate achievement gains to additional expenditures, we conclude 
that an effective compensatory education program will Indeed require 
significant additional resources and we have recomnended as an 
approximation of the needed addition the figure of $300.00. 

To this conclusion, though, we would tenatively add another: there 

Is also an upper boundary of marginal costs, beyond which one would 
probably be wasting money in the application of compensatory resources. 
These conclusions are based on several of the studies we have reviewed. 



The' State evaluations for California, Colorado, Connect- 
icut, and less directly, Wisconsin all indicate a relation- 
ship between costs and effectiveness as costs move up the 
range from $150 to approximately $300 * 



Dr. Herbert Riesling's analysis of successful compensatory 
education programs in California concludes that there is a 
consistent and strong relationship between educational 
resources in the range of $200-$300 and achievement gains 
for disadvantaged children. This Report does not include, 
however, large city observations. 

A recent examination of souie state Title I reports, 
as part of a larger reanalysis of Title I data, concluded that there 
was no compelling evidence for a positive relationship between 
supplementary compensatory education expenditures and achievement 
gains. However, the same organization and authors had identified 
a number of successful compensatory programs Most of these were 



1 / 

2 / 

1 / 



See Appendices B, C, D, 6, 
See Appendix S. 

See Source #77 and Source #75 
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characterised by application of educational resources on a 
concentrated basis* 



At the other end of the scale. Follow Through (at $800 • 

$1 100 additional per pupil jand the More Effective Schools 
program (at over $1,000 additional per pupil) show no evi- 
dence that enormously high resources produce greater incre- 
ments of achievement than programs in the $300 per pupil 
range* 

It must be emphasized that we use the $300 figure as an approximation for 
the in te ns ive approach— we expect variation in the actual amount depend- 
ing on program characteristics. The notion of cost or a dollar amount 
is merely a short expression for the creative use of educational re- 
sources— teachers, paraprof essionals , diagnostic reading instruction 
specialists, individualized curricula, and the like* 

PROGRAM STRATEGY IMPLICATIONS 

Our assessment of the evidence is that compensatory education can help 
disadvantaged children learn and that the chances of success are 
usudly greater beyond the standard effort devoted to basic learning 
programs* Yet we know that money alone provides no guarantee of effec- 
tiveness— there is a continuing intellectual challenge to discover the 
kinds of learning enrichment that can work with different kinds of 
disadvantaged children and there is a need for coherent management, 
innovative planning and vigorous evaluation. 

The state of the evidence which we have reviewed demonstrates a great 
lack of fully reliable, definitive findings, with respect to either 
national, state or local compensatory education efforts* We must: 

(a) Sponsor longitudinal studies of the effects of 
compensatory programs over longer periods of 
time upon individual students; 



(b) Establish controlled experiments which can ' 
determine more carefully than is now the case 
the relationships between program design, 
program costs, program management, the stu- 
dents served and achievement gains* 

Having said we need to know more, much more in the near future we still 
conclude that we know enough now to formulate a program which will try 
to meet the educational needs of millions of disadvantaged children* 
This program has been shaped by our best judgement of evidence: 

« 

we know that compensatory education can be made to work for 
poor children therefore we will use this approach; 
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know that poor children are most in need of educational 
help-therefore we seek to focus on the schools 
which contain substantial proportions of poor 
children; l 
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we know that children need most help with the basic 

skills, reading and aritmethic-therefore we have 
stipulated that the funds be used for this purpose; 

we know that successful programs often require substantial 
departures from typical educational practice and 
usually cost more therefore our provision for about 
$300 per child in compensatory services; 

we know that children learn less effectively when there is 
a great degree of economic or racial isolation- 
therefore our provision for a "transfer bonus" 
which will encourage districts to reduce economic 
and racial concentrations within schools and our 
desire to provide priority in funding for dis- 
tricts which are making efforts to desegregation 
and reduce economic isolation; 

we know that the Federal compensatory, education program has 
not been successful as whole - that funds have not 
reached poor children in the correct pm portion and 
that the formula grant aspects of the program have 
permitted significant amounts of funds to be spent 
in ways which have had only minor educational con- 
sequences for disadvantaged children-therefore we 
seek a project grant program which will permit us to 
attempt a coherent, focused and concentrated com- 
pensatory education effort. 

A well planned and managed project grant program along these lines which 
combines the resources of the Federal Government with the creative 
enthusiasm and sensitivity of local school authorities offers the needed 
assurance that we can hope for some success. 
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In conclusion: a perspective 



The educational aspects of the Equal Educational Opportunities Act are an 
integral part of this Administration's perspective on social policy. In 
his first message on Poverty in 1969, the President told the Nation about 
the negative preliminary results of the Head Start program and he added. 



"This must not discourage us. To the contrary it only demon- 
strates the immense contribution the Head Start program has 
made simply by having raised to prominence on the national 
agenda the fact — known for some time, but never widely recog- 
nized— that the children of the poor mostly arrive at school 
age seriously deficient in the ability to profit from formal 
education. And already significantly behind their contemporaries. 
It also has been made abundantly clear that our schools as they 
now exist are unable to overcome this deficiency." 



Tn Animat 1969 the President submitted the Family Assistance Program to the 
TL for underlying assumption of the President's uelfare reform 
Initiative was that schools could only contribute a part of the resources 
needed*" t^help » children and that improved achievement for these 
childrenwas more°related to family income. This perspective was very 
much shaped by the findings of the Coleman Report. 



A consistent theme of educational policy has been the search for reforms 
which would help poor children. In 1970, the President stated that: 

"The most glaring shortcoming in American education today con- 
tinues to be the lag in essential learning skills in large 
numbers of children of poor families. 



"In the last decade, the Government launched a series of ambi- 
tious, idealistic, and costly programs for the disadvantaged, 
based on the assumption that extra resources would equalize 
learning opportunity and eventually help eliminate poverty. 

"In sane instances, such programs have dramatically improved 
children' 8 educational achievement. In many cases, the pro- 
grams have provided important auxiliary services such as 
medical care and improved nutrition. They may also have 
helped prevent some children from falling even further behind. 

"However, th e best available evide nce indicates that most of 
the compensatory education programs have not meas urably helped 
poor children catch up." (Emphasis in the original) 



1/ Source: 98. 
2/ Source: 97. 



At that time the President concluded that "more of the same," whether in 
programs called compensatory education or in regular school practices 
simply would not provide the effective help needed by poor children. 

A major part of the reform which the President proposed was establishment 
of a National Institute of Education, which is to provide a Federal basis 
for stimulation of educational innovation and the discovery of programs 
and practices that can be effective. 

Similar themes were repeated in the President's recent message on the Equal 
Educational Opportunities Act-routine compensatory and school programs are 
not enough; there is a need for innovative concentrated compensatory efforts 
focused on basic learning skills in order to help poor children. 

Once again the Congress has before it a proposal intended to help millions 
of poor children- -we know that the problems are too great to wait for our 
completely certain knowledge. Responsibility requires that we make our 
best efforts on the basis of the knowledge before us. 



16 



APPENDIX A 




t 



18 



j. 



17 



INTRODUCTION TO THE EVIDENCE 



As in most areas of social policy analysis and research, there are profound 
epistemological problems in the literature dealing with the evidence on 
compensatory education whatever the techniques of analysis. However careful 
the design of a study, there are likely to be different opinions concerning 
its validity, its intrinsic meaning and its relation to governmental policy 
alternatives. 

There is a paradox of first magnitude inherent in most social and educational 
research: if a study is as completely rigorous as might be required to pro- 

vide clear and unmistakable conclusions, its findings will be difficult to 
project into the real world; and, conversely, those analyses which are done 
in the midst of real. school practices are more relevant to policy but more 
likely to be flawed in method and precision of conclusion. But, we add that 
it is possible to endertake policy relevant studies that are carefully de- 
signed and conclusive# 



j. 
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How to Assess Progress in Learning 

The evidence we have summarized is on the whole limited to studies which contain 
pre- and post-test standardized achievement data. This implies a significant 
decision on our part to consider as evidence, positive and negative, a set 
of measures which it has been argued tend to understate the capabilities and 
achievement gains of poor children because of various content and measurement 
biases. He do not feel that standardized tests are the only way to measure 
the achievement of children and we do agree that progress in educational 
measurement is vitally needed so that all economic and cultural groups will 
be fairly assessed. Nevertheless, for the purpose of this analysis we felt 
it essential to limit our consideration to achievement data because it pro* 
vides the only basis for an accumulation of evidence across such a wide 

diversity of studies and is the. only commonly accepted set of siessures now 
available. 

Among those who agree on the use of achievement test data there is a basic 
question concerning the use of "achievement gain" scores as measures for 
improvement. One point of view is that even if a child gains at the rate 
of one year per ypar of instruction but still ends the sixth grade at a 
reading level of 4.5 (instead of 6) he has not really made much progress. 

This is a matter of judgment. The illustration below contrasts the grade 
level attained by children who progress normally and the usual pattern 
for disadvantaged children which results from the typical pattern of cumula- 
tive lag in reading attainment. 
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FIGURE 1: Progress of three children at different rates 

from the same starting point 
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This illustration makes clear that the first step in helping disadvan- 
taged children is to improve the rate at which they learn— as long as that 
remains at the maximum rate of • 7/year for each year of instruction, these 
children fall farther behind each year (examples B and C). For that reason 
we judge improvements above that typical for disadvantaged children in the 
achievement rate to constitute successful compensatory education. 

Clearly, we consider compensatory education to be much more successful and 
effective when the absolute achievement level of the disadvantaged children 
is at or above that typical for their grade. And without question the 
ultimate objective of compensatory efforts should be to help disadvantaged 
children achieve the same reading levels as their classmates. 

Bpoad Issues of Data Validity 

Except for the very carefully controlled, small scale research studies, in 
all the evaluations of compensatory education, we are faced with one or 
another set of data problems which create uncertainty. Let it be noted 
that these uncertainties apply as much to the negative as to the positive 
evaluations of compensatory education. There are two major kinds of 
uncertainties: those pertaining directly to the validity of the achievement 
results; and, those relevant to the nature of the sample from which the 









results are drawn and hence to the generalizations possible as a result 
of any particular study. These are separable issues: a study nay be 
valid for the group of children it has included whether or not that group 
of children is representative of all the children in a given community, 
state or nation, or any category of children such as "disadvantaged 
children." 

Of particular concern in this discussion is the intrinsic validity of the 
studies because we know that there is no evidence available on compensatory 
education which is or could be truly reflective of the United States as 
a whole— and this applies to the national evaluations of Title 1 which 
were based upon achievement data derived from an unrepresentative sub- 
group of children in a sample that was intended to be representative. 

The following is a brief discussion of the effects of the major problems 
associated with large-scale evaluations which use reported data. This 
applies to the two national evaluations of Title 1, the state evaluations 
of Title 1 and all the city evaluations except for the specific sub-studies 
which used control or matched comparison group designs and the same achieve- 
ment tests. 

A. Pre- and Post-test results reported for different children 

Effect : If this happens because of the movement of children, especially 

disadvantaged children in and out of schools during the year, 
then the effect is to understate the real achievement gains 
of children. 

If this is due to conscious substitution of better performing 
children, then of course the effect is to overstate achieve- 
ment gain. 

B. Mixing of different tests as a basis for obtaining overall average gains 

Effect : There are statistical procedures for substantially neutralizing 
the distorting effects of the large variations in the results 
obtained with the same children on different tests. If these 
techniques are not used the effect can be to understate or 
exaggerate the achievement results depending on how the scores 
are derived and overall averages computed. 

C. Use of out of phase tests 

Effect : May understate or overstate achievement gains depending on 

whether a "floor" or "ceiling" test has been used. 

D. Poor test administration and normal clerical errors 

Effect : May either understate.' or overstate gain. 
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E. Regression Coward the mean 

Effect : Due to intrinsic aspects of certain analytical procedures 

and measurement errors (test unreliability) low scoring 
students may score higher on a post-treatment test in the 
absence of any special treatment effect. The extent of 
this effect varies with a number of factors. 

The cumulative weight of such errors can of course be very large. One 
assumes that the states and cities which regularly and systematically 
conduct large scale evaluations of compensatory education take means to 
reduce the errors due to these and other factors. The State of California 
for example used pre-treatment and post -treatment test data in its 1970-71 
analysis only when assured that it was for the same children; and it has 
devised procedures to eliminate error due to mixing of scores from different 
standardized tests and out of phase testing. 



Basis for selection of evidence 

This review is based on all the evidence known to the Department at this 
time. All federally funded evaluations of ESEA Title I and all major 
analyses of other compensatory programs have been included. Most state 
evaluation reports containing achievement test data have been included. 

The state reports which had the most achievement data were analyzed in 
greater depth. The first HEW analysis of state evaluation report achieve- 
ment data was done in 1970 and identified the comparatively few reports with 
substantial amounts of data. 

There are undoubtedly other city or county evaluations of compensatory 
education in addition to those we have included. But those we have 
summarized are the only ones known or available to HEW. The "situational 
evidence" has been selected to provide examples of fairly rigorous 
evaluations of successful compensatory programs. 

Our intention has been to present the major work relevant to the policy 
issues under consideration— it has not been out intention to undertake 
a comprehensive academic survey of everything written about every compen- 
satory education project. 
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CALIFORNIA 

Issue: Does Compensatory Education Work ? 



In California students are usually selected for participation in Title I 
progr ams only if their achievement rate has been less than .7 year's 
growth for each year in school. 1 This means that we can assume a 
maximum of .7 month gain per month of school to be the baseline of compara- 
tive peicfonnance for children receiving compensatory services in that state. 

Using a state-wide testing program with pre -treatment and post-treatment 
comparisons using standardized achievement tests, the state conducts a large 
scale flwmqi evaluation of the effectiveness of Title I compensatory services. 

In the 1970-71 school there were additional efforts to ensure the validity 
of the results and achievement test data for 113, 408 children of the 248.289 
children participating in the reading program met the rigorous criteria. 2 
The analysis indicates that 6l$ of the participating children made achieve- 
ment gains in reading greater than would have been expected without compensatory 
services (see California Table I below). 

CALIFORNIA - TABIE I 



Reading Achievement Gains of ESEA, Title I, Public School Students 
by Grade Level, 1970-71 /2 





Number 

of 


Percent of studcnts» by month's growth 


Grade 


Subs ton tial 


Moderate 


Little or none 


level 


•tudents 


1.5+ 


1.0 to 1 .4 


0.7 to 0.9 


0.6 or less 


One 


10,780 - 


5.2 


19.4 


22.9 


52.5 


two 


14,391 


4.4 


29.7 


32.6 


33.3 


Three 


23,438 


2.5 


14.1 


43.3 


40.1 


Four 


20,236 


10.4 


20.0 


29.7 . 


39.9 


Five 


18,444 


7.3 


13.7 


35.6 


43.4 


Six 


17,S2S 


6.7 


13.4 


43.1 


36.8 


Seven * 


2,185 


13.8 


34.9 


32.5 


18.8 


Eight 


1,838 


13.4 


19.8 


35.0 


31.8 


Fine 


2,725 


21.4 


28.9 


33.6 


16.1 


Ten 


1,267 


17.7 


33.5 


31.0 


17.8 


Eleven 


405 


12.6 


35.1 


38.0 


14.3 


Twelve 


174 


39.1 


33.3 


16.1 


11.5 


Total or 
averag* 


113,408 


7.0 


18.6 


35.5 


38.9 • 






7.0 


54.1 


38.9 



1 Source: 10 

2 Achievement test data was included only when pre and post standardized 
test data on the same children ’Vere available. Also, results were only 
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Of particular note is the fact that 2 (Jf> of all these children showed rates of 
gain above that typical for children who are not disadvantaged. 

In mathematics performance the results are similar. Achievement test data 
for 103,339 students was analyzed and showed that 69$ gained at a rate 
greater than customary for disadvantaged pupils. Of these , 32^ showed ac- 
achievement gains greater than expected for the average pupil (see California 
Table II) . 



CALIFORNIA TABLE II 



Mathematics Achievement Gains of ESEA, 
Title 1, Public School Students 
by Grade Level, 1970-71 /2. 





U„ m La. a C 




Percent of students, 
by month's growth 




viflOe 

level 


number or 
9tudenta 


Substantial 


Moderate 


Little or 






1.5+ 


1.0 to 1.4 


0.7 to 0.9 


none 

0.6 or less 


One 


9,223 


3.2 


16.3 


17.1 


63.4 


Two 


11,836 


7.3 


22.7 


38.2 


31.8 


Three 


20,982 


7.8 


36.2 


33.7 


22.3 


Four 


19,773 


10.1 


35.3 


35.3 


19.3 . 


Five 


17,539 


7.8 


16.1 


50.7 


25.4 


Six 


16,517 


5.3 


13.8 


40.6 


40.3 


Seven 


2,200 


3.6 


15.3 


32.8 


48.3 


Eight 


1,810 


11.4 


2.6 


41.1 


44.9 


Kt« 


2,129 


13.2 


22.3 


25.8 


38.7 


Yen 


859 


19.4 


15.1 


22.0 


43.5 


Eleven 


281 


5.7 


62.6 


— 7.5 


24.2 


Twelve 


170 


64.1 


9.4 * 


1.2 . 


25.3 


Totel 


103,339 


7.6 


24.3 


36.8 


31.3 


Of 

average 




7.6 


61. 


1 


31.3 



included if more than 25 children in a school had taken the tests for a 
specific grade level. 

1 Source :10p. 12. 

2 Source: 10p. 19. 
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Trend over time 



A comparison of reading achievement results over the last four school years 
shows that the proportion of disadvantaged pupils showing improvement above 
the baseline ranged from 52$ in 1967-68 to 67? in 1969-70. (See California. 
Table III). 



CALIFORNIA: TABUS III 



Reading Achievement Gains of ESEA, Title I, Public School Students 

1967-68 Through 1970-71 1 



Year Achievement Results 

(% of pupils) 

Substantial Moderate Little or 
gain gain No gain 

(1.5+) (.7-1.4) (.6 and 

less) 




1 / Source: p. 13 (Percentages prorated to 100$ for those children 
' included in the analysis) 
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Non-public schools 

These positive results also hold for children receiving compensatory 
services in the non-public schools. Reading achievement data was analyzed 
for 4,394 children in the 1970-71 school year and indicated that 68% showed 
gains larger than expected of disavantaged children. More than 32^ of these 
children gained at a rate greater than the expected national norm. In math- 
ematics achievement 69^ of the 3 >5^8 children whose scores were analyzed 
showed gains greater than might have been expected for disadvantaged 
children. 1 (See California Table IV below) 

CALIFORNIA - TABLE IV 

Achievement gains for non-public school pupils: Reading 
and Mathematics. 1970-71 



Percent of students » by months growth 

Number of Substantial Moderate Little or none 

Subject students 1.5+ 1.0 to 1.4 0.7 to 0.9 0.6 or less 

Reading 4,39** 10.4 26.9 30.5 32.2 

Mathematics 3>548 11.5 60.0 29.0 



1 / Source :lOp. 20 (Percentages reflect the proportion of students in 

different gain dategories among those with analyzed 
achievement &afo) • 
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Another type of evidence concerning effectiveness 

Using 1970-71 data the California Department of Education selected a 
sample of schools with "saturated" Title I services - more than 754 of the 
enrolled children were Title I participants. Then a random selection was 
made of a set of schools matching the saturated schools in ethnicity but 
without any children receiving Title I services. The result of a comparison 
of the achievement test results for the two groups of children was that 
they were nearly identical . 1 This finding is very important because it 
is a fact that the children in schools where 7 5% or more of the children 
receive Title I services are the most disadvantaged in the state and the 
comparison group is likely to have contained only a small proportion of 
disadvantaged children. The results suggest that even in this "worst case" 
comparison the saturated compensatory program was able to bring the absolute 
achievement level of the most disadvantaged children up to equality with 
their better-off peers. 

A second difficult case analysis was conducted with the 1970-71 data. 

Picking two grades arbitrarily, results were examined for the children 
who performed least well in the pre -treatment tests (those in the lowest 
quartile + .2GE) . The expectation would be that these children constitute 
a severe test of the efficacy of compensatory education. The results in- 
dicate that significant proportions of these children have made gains above 
.7- -note, however, that none in the third grade made substantial gains 
(see table California 5). 
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CALIFORNIA TABLE V 

Achievement Gain Rates for Children in the Lower Q uartile 
of Initial Performance --Reading. 1970-71 







Number 


Proportions in achievement gain groups 


Grade 


Test 


Analyzed 


Substantial 

(1.5+) 


Moderate 


Little or none 
(.6- ) 


3 


CTBS 


1,000 


-- 


50-65% 


30-40% 


3 


Stanford 


3,500 


— 


65% 


35% 


6 


CTBS 


3,000 


8% 


67% 


25% 



Source: Analysis provided the Department of Health, Education and 

Welfare by the California State Department of Education, 
working papers. 



1/ Specifically a mean raw score of 45 for the third grade in the saturated 
” schools and 46 for the children in the matched schools. Sources: 
print -out from SEA. \ 
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Is there a relationship between cost and probability of success? 



Since the 196 9-70 school year California has modified its Title I regu- 
lations to emphasize that services should be concentrated at the level 
of at least $300 per child. * This formalized a preference that has been 
evident in the state department of education at least since the 1967-68 
school year. Dr. Alexander Law, Chief of Program Evaluation for the 
State, reports that for the 1970-71 school year he has audit and other 
evidence suggesting that local districts have in fact allocated their funds 
in accordance with he $300 per child guideline and that there is little 
likelihood of a large variation in supplementary per pupil expenditures 
above that level because most districts prefer to reach as many children 
as possible. 

This strongly suggests that the California Title I program for 1970-71 
can be considered a fairly good test of the intensity hypothesis. Under 
these assumptions, our judgment is that the high proportions of children 
making improved gains in reading and mathematics strongly suggests that 
the strategy of concentrating resources at some "critical mass" level is 
a sensible one. This does not mean that this is sufficient condition for 
success as is indicated by the 40% of the children who made no improved 
gains in reading and the 31% showing no improvement in mathematics in 
1970-71. 

The average $300 per child in Title I funds was not all used for basic 
learning programs in reading and mathematics. It is in fact not possible 
to separate this aggregate figure into its components on a school by 
school basis and thereby derive a direct correlation with mean achievement 
test scores by school. However, an average of supplementary funds for 
reading and mathematics for the Title I participants has been calculated 
which includes Title I and other compensatory aid sources: in 1970-71 

the average child received $242 in compensatory reading services and $140 
in mathematics compensatory services. This average amount for reading 
reinforces the notion of a "critical mass" while the lesser amount for 
mathematics in combination with the positive performance results shows 
that the intensity needed may vary from subject to subject. 

Questions about the cost data 

The FY 1969 California state report concluded that there seemed to be a 
strong and clear relationship between' cost and achievement gains in com- 
pensatory programs. As a result the U. S. Office of Education sent two 
individuals to consult with state officials: they collaborative ly pre- 

pared a quickly done analysis of 709 Title I projects where cost and 
achievement data were both available. A brief summary of that joint 
analysis follows: 



1/ Source: p. 3. 

2 J See, passim Sources: 

3/ This is the California study cited by the President in his message on 
EEOA, March 17, 1972. -V.2& 



